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SPECIFICATION 

METHOD AND APPARATUS FOR EXTENDING VIDEO CONTENT ANALYSIS 

TO MULTIPLE CHANNELS 

5 BACKGROUND OF THE INVENTION 

Field of the Invention 

The invention relates to multiple channel video 
...^^ systems, such as security systems and more particularly to 

JjLO such systems that perform video content analysis. 

i3 Backgrotmd 

In many instances, it is useful for video to be 
analyzed automatically by a computer system rather than 
'As having a person watch the video. For example, in a 

security system, a human observer is not likely to be 
sufficiently observant to catch a sudden change in a scene 
which remains changeless for hours at a time. Also, it is 
useful to have multiple scenes interpreted by a single 
2 0 observer. For example, one security guard may observe the 
goings on in multiple scenes captured by different cameras. 

In domestic applications, video content from 
multiple sources, such as multiple channels, may be 
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received and analyzed to automatically recommend changing a 
channel. For example, the printed name of a favorite actor 
might be found on one channel while the user is watching 
another channel . 

The general field of video content analysis is a 
broad one involving many different motives for analyzing 
video content. But, generally speaking, "simple" 
surveillance devices such as video motion detectors and 
VCRs are designed and built to work with multiple channels 
of input. This decreases the amount of equipment a user 
has to buy and offers greatly increased value for money. 
However, more sophisticated video processing, such as that 
available as PC software, or as high-end motion detector 
modules, is designed to work on a single channel at a 
single time. For example, in advanced security systems, 
content analysis devices are designed to work on one 
channel at a time, requiring the users to purchase multiple 
devices, one for each "analyzed" channel. 

Referring to Fig. 1, in a typical advanced 
security system, for example, video data is received from a 
video source 100 which may be, for example, data from a 
camera. A piece of ancillary equipment 195, such as a 
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video recorder, may be somewhere in the data loop. A 
content analysis process 110 receives the data (perhaps in 
parallel or in series as indicated) analyzes it and 
presents results 12 0 to some other process, for example, an 
5 alarm process. 

One system can be used to analyze multiple 
channels by scanning in a round robin fashion, but real 
time information may be lost by doing that, such as 
calculated data that rely on motion data. A principal 
i.[10 cause of this delay is that fact that each analogue video 

signal needs to be digitized before it can be processed 
in digitally. Switching from one signal to the next may incur 
^^^3 a delay of up to one frame time while the digitizer 
I"": attempts to synchronize with the sync of the new source, 
i'ls Thus for four video signals, the frame rate is not just 

four times less than for one signal; it may be as much as 
eight times less. Also, if an intervening piece of 
ancillary equipment, such as a video recorder, is in the 
loop, the rotating of multiple channels through that loop 
20 could make the data unusable for purposes of that ancillary 
equipment . 
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There exists a need in the art for ways of 
providing content analysis without the need for the 
purchase of multiple systems for providing content analysis 
and with provision for correct real time information. 

5 

SUMMARY OF THE INVENTION 

A video content analysis system extends content 
analysis capability of one system to multiple channels by 
;^ providing for the spatial multiplexing of the multiple 
i^O channels and appropriately analyzing the spatially 

multiplexed video signal. The resulting system may be 
Ln lower in cost that present systems and permit the system to 

Is 

^•^ work with ancillary equipment such as video recorders. The 

•! '-pr 

system also preserves the real-time information inherent in 
the multiple source signals. 

According to an embodiment, the invention 
provides a method of analyzing content in video data, 
comprising the steps of multiplexing the video data such 
that video of multiple scenes are distributed in a single 
20 video stream, at least part of each of the video data being 
apportioned to a respective part of a moving image defined 
by a resulting multiplexed moving image, analyzing content 
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of the multiplexed video image such that data in others of 



the each of the video data is ignored to produce an 



analysis particular to one of the multiple scenes. 



According to another embodiment, the invention 



5 provides a method of analyzing multiple video channels, 
comprising the steps of multiplexing multiple video data 
sets at the multiplexer to produce a spatially multiplexed 
moving image, analyzing at least a first portion of the 
□ spatially multiplexed moving image, the first portion 

WIO corresponding to a first of the channels, the step of 

ry 

analyzing including ignoring data in the multiplexed moving 
i"^ image corresponding to channels other than the first of the 
r1 channels. 



Cl5 provides a device for analyzing video content on multiple 
channels, comprising an input adapted to receive spatially 
multiplexed video data, a controller programmed to select 
spatially distinct portions of the. video data, each of the 
portions respective of a particular video data channel, the 
2 0 controller being further programmed to analyze content of 
the spatially distinct portions such that data from one 
does not interfere with the analysis of another. 



According to an embodiment, the invention 
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The invention will be described in connection 



with certain preferred embodiments, with reference to the 
following illustrative figures so that it may be more fully 
understood. With reference to the figures, it is stressed 
that the particulars shown are by way of example and for 
purposes of illustrative discussion of the preferred 
embodiments of the present invention only, and are 
presented in the cause of providing what is believed to be 
the most useful and readily understood description of the 
principles and conceptual aspects of the invention. In 
this regard, no attempt is made to show structural details 
of the invention in more detail than is necessary for a 
fundamental understanding of the invention, the description 
taken with the drawings making apparent to those skilled in 
the art how the several forms of the invention may be 
embodied in practice. 



analysis system according to the prior art. 

Fig. 2 is a visual representation of a spatially 
multiplexed video data stream for illustration purposes. 



BRIEF DESCRIPTION OF THE DRAWINGS 



Fig. 1 is an illustration of a video content 
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Fig. 3 is a diagram of data flow according to an 
embodiment of the invention. 

Fig. 4 is a diagram of an example hardware 
environment that could be used to implement the invention. 

5 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Referring to Fig. 2, the invention involves 
;j spatial multiplexing of a video signal. One simple type of 
rULO spatial multiplexing is to shrink each image from multiple 
'2 channels to a different part of a single image. Here, four 
' channels of video are incorporated in respective frames 

150, 155, 160, and 165. Although it possible to. spatially 
I J multiplex the image in any arbitrary fashion, for example 
\,A5 it may be viewed simply as arranging pixels from multiple 

subsampled scenes in different places in a single frame, it 
is preferred to do it in a way that makes the resulting 
image sensible to an observer. For example, when ancillary 
equipment is used to record or transfer the multiplexed 
20 video stream, the result may still be usable for its 

intended purpose. Such an arrangement will typically mean 
that adjacent pixels from each scene will be adjacent in 
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the composite image. This arrangement is also typically 
easier to process on a computer, because the composite 
image will be loaded into memory and adjacent pixels will 
be in adjacent memory locations, and cache memory can be 
used effectively. 

Referring now to Fig. 3, a system for 
implementing the invention includes various video data 
sources 101, 102, 103, and 104. Although the number of 
sources shown is four, the number is quite arbitrary. The 
video data is applied to a spatial multiplexer 270. The 
latter combines the video data, preferably in a way that 
makes the video data intelligible when reproduced by 
conventional equipment. Thus, for example, showing the 
video as spatially separate images in each frame of the 
multiplexed image with each frame of the source 
corresponding to a frame of the multiplexed video is 
preferred. An example is shown in Fig. 2. 

The multiplexed image may be applied to ancillary 
equipment 3 95 such as a video recorder, broadcast system, 
display, or other device that reproduces or transforms the 
video data conventionally. The ancillary equipment 395 is 
shown in series, but it could just as easily be connected 
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in parallel or at some other point in the system. A 
demultiplexer 275 also receives the multiplexed video 
signal from the multiplexer 270. The demultiplexer may be 
a software front end to a software process that analyzes 
the multiple channels of video data. Alternatively, it may 
be a hardware device that outputs the separate video data 
on multiple physical channels. Basically, the 
demultiplexer 275 allocates the data in the multiplexed 
stream to respective content analysis processes 211, 212, 
213, and 214 so that the video data can be analyzed without 
the interference of data from the other sources 101-104. 
Note that certain analysis process, such as motion 
detection, can be performed on the raw multiplexed image. 
Each independent process 211-214 may produce a respective 
result 221, 222, 223, and 224. 

Note that the processes of Fig. 3 may be 
generated on a single piece of hardware such as a computer 
or embedded system. They may be achieved by multitasking 
or multithreading or any suitable software technique. Data 
may be shared among them depending on the type of analysis 
done. Also, another content analysis process (not shown) 
may be added which operates on the multiplexed video data 
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permitting the separate content analysis processes (211- 
214) to avoid having to perform that process or that 
process might feed data to the separate content analysis 
processes 211-214. For example, such a process might 
perform motion detection on the multiplexed data. 

Referring now to Fig. 4, an example hardware 
environment that may be used to support the processes of 
the invention includes cameras 310 that receive video input 
from multiple respective scenes. A computer 34 0, which may 
be an embedded system or analog multiplexer receives the 
video data and multiplexes it. The multiplexed image may 
be applied in series or parallel to ancillary equipment 390 
as discussed above. The multiplexed video data may be 
transmitted over a network, the Internet 330. 
Alternatively, the data may be transferred over an analog 
line or a switched network such as a telephone network. 
The data may be received by additional ancillary equipment 
3 95 and demultiplexed by a suitable system, for example a 
computer 350. The latter may also perform content 
analysis . 

It will be evident to those skilled in the art 
that the invention is not limited to the details of the 
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foregoing illustrative embodiments, and that the present 
invention may be embodied in other specific forms without 
departing from the spirit or essential attributes thereof. 
The present embodiments are therefore to be considered in 
all respects as illustrative and not restrictive, the scope 
of the invention being indicated by the appended claims 
rather than by the foregoing description, and all changes 
which come within the meaning and range of equivalency of 
the claims are therefore intended to be embraced therein. 



